Papers with motion representations
Video-guided Machine Translation with Spatial Hierarchical Attention Network (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing studies use pretrained motion detection models as verb sense ambiguity representations to solve the verb sense problem. |
| Approach: | They propose to use video contents as auxiliary information to address the word sense ambiguity problem in machine translation. |
| Outcome: | Experiments on the VATEX dataset show that the proposed system achieves 35.86 BLEU-4 score, which is 0.51 score higher than the single model of the SOTA method. |
Three Stream Based Multi-level Event Contrastive Learning for Text-Video Event Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for event extraction ignore motion representations in videos and are misguided by background noise. |
| Approach: | They propose a text-video based multimodal event extraction framework that integrates video appearance features and motion representations with video appearance. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods in the event extraction field. |